Papers with human evaluation tasks
Modelling Variability in Human Annotator Simulation (2024.findings-acl)
Copied to clipboard
| Challenge: | Human annotator simulation (HAS) is a cost-effective alternative to human evaluation tasks. |
| Approach: | They propose a framework to model human annotation variability via meta-learning . conditional softmax flow model leverages diverse human annotations via meta learning . results demonstrate that method can predict aggregated behaviours of human annotators . |
| Outcome: | The proposed method achieves state-of-the-art performance on two real-world human evaluation tasks: emotion recognition and toxic speech detection. |